Monitoring industrial equipment using audio

ABSTRACT

Systems, methods, and devices for monitoring industrial equipment using audio are described herein. One system includes two computing devices. The first computing device can receive, from an audio sensor, audio sensed during operation of industrial equipment, extract a plurality of features from the audio, determine whether any portion of the audio is anomalous, and send, upon determining a portion of the audio is anomalous, the anomalous portion of the audio to the second, remotely located, computing device. The second computing device can provide the anomalous portion of the audio to a user to determine whether the anomalous portion of the audio corresponds to a fault occurring in the equipment, and receive, from the user upon determining the anomalous portion of the audio corresponds to a fault occurring in the equipment, input indicating the anomalous portion of the audio corresponds to the fault to learn fault patterns in the equipment.

The present application is a continuation of U.S. application Ser. No.16/033,883, filed Jul. 12, 2018, entitled, “MONITORING INDUSTRIALEQUIPMENT USING AUDIO,” which is hereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates generally to systems, methods, anddevices for monitoring industrial equipment using audio.

BACKGROUND

A heating, ventilation, and air conditioning (HVAC) system can be usedto control the environment within a facility (e.g., building), and isone example of an industrial system that may be associated with thefacility. For example, an HVAC system can include a number of components(e.g., equipment) that can be used to control the air temperature ofdifferent zones (e.g., rooms, areas, spaces, and/or floors) of afacility, in order to keep the zones in a comfort state for theiroccupants. As an additional example in which the facility is a retailfacility, the HVAC equipment may be used to continuously operate (e.g.,cool) commercial refrigerators and/or freezers.

During operation of an industrial system such as an HVAC system,however, faults in the system (e.g., in the equipment of the system) maysometimes occur. Detecting and correcting faults in the equipment of thesystem can be important to provide and maintain a comfortableenvironment for the occupants of the facility, to prevent the fault fromcausing further damage to the system, to prevent downtime in theequipment that may lead to significant loss, and/or to avoid inefficientoperation of the system which may result in higher energy consumption,for example.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a system for monitoring industrialequipment using audio in accordance with an embodiment of the presentdisclosure.

FIG. 2 illustrates a conceptual example of a method for monitoringindustrial equipment using audio in accordance with an embodiment of thepresent disclosure.

FIG. 3 illustrates an example display of an alert of an anomalous audioportion provided to a user in accordance with an embodiment of thepresent disclosure.

FIG. 4 illustrates an example of a computing device for monitoringindustrial equipment using audio in accordance with an embodiment of thepresent disclosure.

DETAILED DESCRIPTION

Systems, methods, and devices for monitoring industrial equipment usingaudio are described herein. For example, an embodiment includes twocomputing devices. The first computing device can receive, from an audiosensor, audio sensed by the sensor during operation of industrialequipment, extract a plurality of features from the audio, determinewhether any portion of the audio is anomalous, and send, upondetermining a portion of the audio is anomalous, the anomalous portionof the audio to the second, remotely located, computing device. Thesecond computing device can provide the anomalous portion of the audioto a user to determine whether the anomalous portion of the audiocorresponds to a fault occurring in the equipment, and receive, from theuser upon determining the anomalous portion of the audio corresponds toa fault occurring in the equipment, input indicating the anomalousportion of the audio corresponds to the fault to learn fault patterns inthe equipment.

Previous approaches for monitoring industrial equipment, such as, forinstance, equipment of an HVAC system, may use vibration sensors todetect faults occurring in the equipment. However, in order toeffectively detect faults, such vibration sensors need to be in directcontact with (e.g., attached and/or mounted to) the equipment, whichresults in additional downtime being needed for calibration and/orrepair of the sensors. Accordingly, such vibration sensors may beconsidered invasive sensors relative to the HVAC system. Further, suchvibration sensors may only be usable with certain types of HVACequipment (e.g., such vibration sensors may be equipment specific).Accordingly, such vibration sensors may not be considered agnosticrelative to the equipment of the HVAC system.

In contrast, embodiments of the present disclosure may use audio sensorsto detect faults occurring in industrial (e.g., HVAC) equipment. Suchaudio sensors may not need to be in direct contact with the equipment toeffectively detect faults in the equipment, and therefore may beconsidered non-invasive sensors relative to the HVAC system. Further,such audio sensors may be usable with all types of industrial equipment(e.g., may not be equipment specific), and therefore may be consideredagnostic relative to the equipment. Further, such audio sensors may becheaper and/or easier to install than vibration sensors.

Further, previous approaches for monitoring industrial equipment todetect faults may be reactive. For instance, in previous approaches, anyaction needed to correct a fault may be taken only after the fault hasbeen detected, which may result in unpredictable equipment downtimeand/or manual effort by the technician to correct the fault. Further,the technician may only be able to assess and correct the fault upon amanual, on-site inspection, which increases the amount of time needed tocorrect the fault, and adversely affects the technician's productivity.Further, in some instances it may be difficult for the technician toassess and correct the fault quickly and/or accurately due to, forinstance, a lack of knowledge and/or skill on the part of thetechnician.

In contrast, embodiments of the present disclosure may be able toproactively detect and correct faults occurring in industrial (e.g.,HVAC) equipment. For instance, monitoring the equipment using audio mayprovide an early indication of a fault occurring in the equipment (e.g.,before the fault causes a significant problem), such that the fault maybe detected and/or corrected with minimal or no downtime in theequipment. Further, embodiments of the present disclosure may be able toautomatically detect and/or correct a fault occurring in the equipment(e.g., the fault may be detected and/or corrected without a technicianhaving to visit the site and manually inspect the equipment). Further,embodiments of the present disclosure may facilitate the learning offault patterns over time via crowd sourcing for identification ofanomalous audio, such that future faults can be accurately detected andcorrected across multiple facilities in a quicker and more efficientmanner. For instance, embodiments of the present disclosure may not haveto be pre-configured; rather, embodiments can be taught to recognizeunique patterns for the environment of the system. Further, embodimentsof the present disclosure can self-learn the noise profile and/orcharacterization of the equipment upon deployment. Further, embodimentsof the present disclosure can facilitate the capture and distribution ofexpert technician knowledge across similar equipment in differentfacilities.

In the following detailed description, reference is made to theaccompanying drawings that form a part hereof. The drawings show by wayof illustration how one or more embodiments of the disclosure may bepracticed.

These embodiments are described in sufficient detail to enable those ofordinary skill in the art to practice one or more embodiments of thisdisclosure. It is to be understood that other embodiments may beutilized and that mechanical, electrical, and/or process changes may bemade without departing from the scope of the present disclosure.

As will be appreciated, elements shown in the various embodiments hereincan be added, exchanged, combined, and/or eliminated so as to provide anumber of additional embodiments of the present disclosure. Theproportion and the relative scale of the elements provided in thefigures are intended to illustrate the embodiments of the presentdisclosure, and should not be taken in a limiting sense.

The figures herein follow a numbering convention in which the firstdigit or digits correspond to the drawing figure number and theremaining digits identify an element or component in the drawing.Similar elements or components between different figures may beidentified by the use of similar digits.

As used herein, “a” or “a number of” something can refer to one or moresuch things, while “a plurality of” something can refer to more than onesuch things. For example, “a number of faults” can refer to one or morefaults, while “a plurality of faults” can refer to more than one fault.Additionally, the designators “N”, “M”, and “P”, as used herein,particularly with respect to the reference numerals in the drawings,indicates that a number of the particular feature so designated can beincluded with a number of embodiments of the present disclosure.

FIG. 1 illustrates an example of a system 100 for monitoring industrialequipment using audio in accordance with an embodiment of the presentdisclosure. The industrial equipment can be, for example, the equipmentof a heating, ventilation, and air conditioning (HVAC) system of afacility (e.g., building), such as, for instance, an office building(e.g., a commercial office building), or a retail facility (e.g., aretail facility having commercial refrigerators and/or freezers).However, embodiments of the present disclosure are not limited to aparticular type of facility, or to a particular type of industrialequipment. For instance, embodiments of the present disclosure can beused in a process plant system, conveyor belt system, or any other typeof industrial setting that generates noise that can be captured.

The HVAC system can be used to control the environment within thefacility. For example, the HVAC system can include a number ofcomponents (e.g., equipment 102-1, 102-2, . . . , 102-N illustrated inFIG. 1) that can be used to control the air temperature of differentzones (e.g., rooms, areas, spaces, and/or floors) of the facility, inorder to keep the zones in a comfort state for their occupants. As anadditional example, the equipment of the HVAC system may be used tocontinuously operate (e.g., cool) commercial refrigerators and/orfreezers of the facility.

The equipment of the HVAC system (e.g., equipment 102-1, 102-2, . . . ,102-N) can include, for example, valves such as heating and/or coolingvalves, chillers (e.g., chiller plant), boilers (e.g., boiler plant),pumps such as hot water and/or chilled water pumps, fans, compressors,air dampers such as variable air volume (VAV) dampers, air handlingunits (AHUs) (e.g., AHU plant), coils such as heating and/or coolingcoils, air filters, and/or cooling towers, among other equipment. TheHVAC system may also include connections (e.g., physical connections)between the equipment, such as a chain of equipment (e.g., duct work,pipes, ventilation, and/or electrical and/or gas distribution equipment)that connects the components of the HVAC system, among otherconnections.

As shown in FIG. 1, system 100 can include a number of audio sensors104-1, 104-2, . . . , 104-M located within the facility. Audio sensors104-1, 104-2, . . . , 104-M can be, for example, microphones. However,embodiments of the present disclosure are not limited to a particulartype of audio sensor.

Audio sensors 104-1, 104-2, . . . , 104-M can sense (e.g., collectand/or record) audio (e.g. noise) that occurs during (e.g., resultsfrom) the operation of HVAC equipment 102-1, 102-2, . . . , 102-N. Forinstance, audio sensor 104-1 can sense the audio that occurs during theoperation of equipment 102-1, audio sensor 104-2 can sense the audiothat occurs during the operation of equipment 102-2, and audio sensor104-M can sense the audio that occurs during the operation of equipment102-N. However, embodiments of the present disclosure are not limited tosuch a one-to-one correspondence between the equipment and audio sensors(e.g., in some embodiments, one audio sensor may sense the audio thatoccurs during the operation of a plurality of equipment). Audio sensors104-1, 104-2, . . . , 104-M can sense the audio as audio clips (e.g., 10second audio clips) continuously over a period of days, in order toencapsulate the typical operation noise profile for the equipment.

As an additional example, audio sensors 104-1, 104-2, . . . , 104-M cansense audio that occurs during an event associated with the HVAC system.For instance, audio sensors 104-1, 104-2, . . . , 104-M can sense audioduring an event associated with (e.g., that occurs during) operation ofHVAC equipment 102-1, 102-2, . . . , 102-N. Examples of such events willbe further described herein.

Audio sensors 104-1, 104-2, . . . , 104-M may not be in direct contactwith HVAC equipment 102-1, 102-2, . . . , 102-N. For instance, audiosensors 104-1, 104-2, . . . , 104-M may not be attached or mounted toequipment 102-1, 102-2, . . . , 102-N. As such, audio sensors 104-1,104-2, . . . , 104-M may be non-invasive, equipment agnostic sensors.

As shown in FIG. 1, system 100 can include computing device 106.Computing device 106 can be located at (e.g., within) the facility. Anexample of such a computing device will be further described herein(e.g., in connection with FIG. 4). For instance, such a computing devicecan include a memory, a processor, and a user interface, as will befurther described herein (e.g., in connection with FIG. 4).

As shown in FIG. 1, computing device 106 can receive the audio sensed byaudio sensors 104-1, 104-2, . . . , 104-M from the audio sensors. Forinstance, computing device 106 can receive the audio sensed by the audiosensors during the operation of HVAC equipment 102-1, 102-2, . . . ,102-N, and determine whether any portion (e.g., clip) of the audiosensed during the operation of the equipment is anomalous. For instance,computing device 106 can determine whether any portion of the audio isan outlier from the normal operation of the equipment. Computing device106 can make this determination in a continuous (e.g., 24/7),unsupervised manner (e.g., automatically, without using or needing userinput at the facility). For instance, computing device 106 can establishthe normal operational noise profile for the equipment to determine theboundaries of normal operation, which can then be used to automaticallydetect anomalous equipment noise. Further, computing device 106 can bean edge device that can perform the detection of the anomalous equipmentnoise using unsupervised machine learning as part of edge analytics.

For example, computing device 106 can determine whether any portion ofthe audio sensed by the audio sensors 104-1, 104-2, . . . , 104-M duringthe operation of the equipment 102-1, 102-2, . . . , 102-N is anomalousby comparing the sensed audio to the profile (e.g., signature and/orpattern) of the audio sensed by the audio sensors during normaloperation of the equipment. A portion of the sensed audio may bedetermined to be anomalous upon the comparison indicating that portionof the audio deviates from the profile (e.g., is outside the boundariesof the profile) of the audio sensed during the normal operation of theequipment. Such a deviation can be flagged as an anomaly, and thatanomalous portion (e.g., the corresponding audio clip) can be stored bycomputing device 106 (e.g., for future play back to a technician toprioritize and/or plan equipment inspection and/or maintenance).

Computing device 106 can generate (e.g. identify) the profile of theaudio sensed by the audio sensors 104-1, 104-2, . . . , 104-M during thenormal operation of the equipment 102-1, 102-2, . . . , 102-N byextracting a plurality of features from the audio sensed during thenormal operation of the equipment, and combining the plurality ofextracted features to determine the boundaries for the profile. Theplurality of extracted features can include, for example, time-basedfeatures (e.g., time statistics) and/or frequency-based features (e.g.,frequency statistics) of the audio sensed by the audio sensors duringthe normal operation of the equipment. For example, the audio sensorsmay capture the aggregate operational audio signature of the equipmentusing the plurality of features extracted from a wideband frequencyrange, which may reflect specific artifacts in the sensed audio. Basedon this signature, computing device 106 can determine (e.g., learn) theboundaries for the normal operation of the equipment as a combination ofthe audio artifacts. In such a manner, computing device 106 canself-learn the noise profile of the equipment, in order to characterizethe normal operational noise of the equipment in its deployedenvironment.

As shown in FIG. 1, system 100 can include an additional computingdevice 110. Computing device 110 can be located remotely from thefacility (e.g., remotely from computing device 106). For instance,computing device 110 can be part of a centralized, cloud-based analyticsservice (e.g., servers and/or databases). An example of such a computingdevice will be further described herein (e.g., in connection with FIG.4). For instance, such a computing device can include a memory, aprocessor, and a user interface, as will be further described herein(e.g., in connection with FIG. 4).

Upon determining a portion of the audio sensed by audio sensors 104-1,104-2, . . . , 104-M is anomalous, computing device 106 can send (e.g.,transmit) the anomalous portion of the audio to computing device 110(e.g., computing device 110 can receive the anomalous portion of theaudio from computing device 106). However, portions of the sensed audionot determined to be anomalous may not be sent to computing device 110by computing device 106 (e.g., only anomalous audio portions may be sentto computing device 110 by computing device 106).

As shown in FIG. 1, computing device 106 can send the anomalous portionof the audio to computing device 110 via network 108 (e.g., computingdevice 110 can receive the anomalous portion of the audio from computingdevice 106 via network 108). Network 108 can be a wired or wirelessnetwork. For example, network 108 can be a network relationship throughwhich computing devices 106 and 110 can communicate. Examples of such anetwork relationship can include a distributed computing environment(e.g., a cloud computing environment), a wide area network (WAN) such asthe Internet, a local area network (LAN), a personal area network (PAN),a campus area network (CAN), or metropolitan area network (MAN), amongother types of network relationships. For instance, the network caninclude a number of servers that receive the anomalous audio portionfrom computing device 106, and transmit the anomalous audio portion tocomputing device 110 via a wired or wireless network.

As used herein, a “network” can provide a communication system thatdirectly or indirectly links two or more computers and/or peripheraldevices and allows users to access resources on other computing devicesand exchange messages with other users. A network can allow users toshare resources on their own systems with other network users and toaccess information on centrally located systems or on systems that arelocated at remote locations. For example, a network can tie a number ofcomputing devices together to form a distributed control network (e.g.,cloud).

A network may provide connections to the Internet and/or to the networksof other entities (e.g., organizations, institutions, etc.). Users mayinteract with network-enabled software applications to make a networkrequest, such as to get a file or print on a network printer.Applications may also communicate with network management software,which can interact with network hardware to transmit information betweendevices on the network.

Computing device 110 can provide the received anomalous audio portion toa user (e.g., technician) to determine whether the anomalous audioportion corresponds to a fault occurring in the equipment. For example,computing device 110 can provide an alert to the user of the anomalousaudio portion and play the anomalous audio portion for the user, who canlisten to it and determine whether it corresponds to a fault occurringin the equipment. An example of such an alert will be further describedherein (e.g., in connection with FIG. 3).

As used herein, a fault occurring in the equipment can include and/orrefer to the equipment functioning improperly and/or causing abnormalbehavior in the HVAC system and/or facility, and/or to an event thatoccurs to cause the equipment to function improperly or cause theabnormal behavior. For example, a fault in the equipment can includeand/or refer to the equipment breaking down, malfunctioning, ceasing tooperate correctly, or operating in an unexpected manner. As anadditional example, a fault can include and/or refer to abnormal (e.g.,anomalous) behavior of the equipment.

Upon the user determining the anomalous audio portion corresponds to afault occurring in the equipment, computing device 110 can receive fromthe user an input indicating that the anomalous audio portioncorresponds to the fault. For instance, the user may identify (e.g.,label, annotate, and/or validate) the anomalous audio portion ascorresponding to the fault. Further, the user may add comments and/orremarks about the fault in the form of text input.

As such, computing device 110 can learn that the anomalous audio portioncorresponds to the fault (e.g., that particular type of fault) in a user(e.g., technician) supervised manner, and store the anomalous audioportion accordingly. For example, computing device 110 can usesupervised machine learning to classify anomalous audio as faults, andbuild an audio corpus (e.g., database of audio files) for differentclasses of faults. This process can be referred to herein as “trainingmode”.

Computing device 110 can then use this knowledge to detect (e.g., in anunsupervised manner) subsequent faults that may occur in HVAC equipment102-1, 102-2, . . . , 102-N. As such, computing device 110 can learnfault patterns in the equipment over time via crowd sourcing foridentification of anomalous audio, such that computing device 110 canaccurately detect subsequent (e.g., future) faults in the equipment in aquicker and more efficient manner, without having to be pre-configured.This process can be referred to herein as “test mode”.

For example, during subsequent operation of HVAC equipment 102-1, 102-2,. . . , 102-N, audio sensors 104-1, 104-2, . . . , 104-M can sense theaudio that occurs, and send this additional audio to computing device106 (e.g., computing device 106 can receive the additional audio fromthe audio sensors). Computing device 106 can then determine whether anyportion of this additional audio is anomalous, in a manner analogous tothat previously described herein for the previously sensed audio. Upondetermining a portion of the additional audio is anomalous, computingdevice 106 can send the anomalous portion of the additional audio tocomputing device 110 (e.g., computing device 110 can receive theanomalous portion of the additional audio from computing device 106) ina manner analogous to that previously described herein for thepreviously sensed audio.

Computing device 110 can then determine whether the anomalous portion ofthe additional audio corresponds to a fault occurring in the equipmentbased, at least in part, on the input previously received from the user(e.g., the input indicating the previous anomalous audio portioncorresponded to a fault). For example, computing device 110 can comparethe anomalous portion of the additional audio (e.g., the profile of theadditional anomalous portion of the additional audio) to the previousanomalous audio portion (e.g., to the profile of the previous anomalousaudio portion) indicated by the user to correspond to a fault todetermine whether the anomalous portion of the additional audio matchesthe previous anomalous audio portion. If the comparison indicates amatch, computing device 110 can determine that that the anomalousportion of the additional audio corresponds to a fault (e.g., the sametype of fault that the previous anomalous audio portion correspondedto).

Upon determining the anomalous portion of the additional audiocorresponds to the fault (e.g., if the comparison indicates a match),computing device 110 can provide an alert to the user of computingdevice 110 indicating the fault is occurring. If, however, thecomparison does not indicate a match (e.g., thereby indicating that theanomalous portion of the additional audio does not correspond to thesame type of fault that the previous anomalous audio portioncorresponded to), computing device 110 can provide the anomalous portionof the additional audio to the user to determine whether the anomalousportion of the additional audio corresponds to a fault (e.g., adifferent type of fault) occurring in the equipment, in a manneranalogous to that previously described herein for the previous anomalousaudio portion.

Upon the user determining the anomalous portion of the additional audiocorresponds to a fault occurring in the equipment, computing device 110can receive from the user an input indicating that the anomalous portionof the additional audio corresponds to the fault, in a manner analogousto that previously described herein for the previous fault. As such,computing device 110 can learn that the anomalous portion of theadditional audio corresponds to the fault, and also store this anomalousaudio portion accordingly. Computing device 110 can then use thisadditional knowledge in an analogous manner to detect subsequent faultsthat may occur in HVAC equipment 102-1, 102-2, . . . , 102-N. Thisprocess can continue to be performed throughout the operation of HVACequipment 102-1, 102-2, . . . , 102-N, such that computing device 110can continue to learn and detect additional faults (e.g., additionaltypes of faults) during the operation of the equipment in a proactivemanner (e.g., such that computing device 110 can continue to learn faultpatterns in the equipment over time via crowd sourcing foridentification of anomalous audio, such that computing device 110 cancontinue to accurately detect future faults in the equipment in aquicker and more efficient manner, without having to be pre-configured).

As an additional example, computing device 106 can receive the audiosensed by audio sensors 104-1, 104-2, . . . , 104-M during an eventassociated with (e.g., that occurs during) the operation of HVACequipment 102-1, 102-2, . . . , 102-N, and send, via network 108, thisaudio to computing device 110. Computing device 110 can provide thisaudio to a user (e.g., technician) to determine whether the eventcorresponds to a fault occurring in equipment 102-1, 102-2, . . . ,102-N. For example, computing device 110 can provide an alert of theevent to the user and play the audio sensed during the event for theuser, who can listen to it and determine (e.g., based on the user'sexpert knowledge) whether it corresponds to a fault occurring in theequipment.

Upon the user (e.g., technician) determining the event corresponds to afault occurring in the equipment, computing device 110 can receive fromthe user an input indicating that the audio sensed during the eventcorresponds to the fault. For instance, the user may identify (e.g.,annotate) the audio sensed during the event as corresponding to thefault, and store this identified audio (e.g., the profile of the audio)accordingly.

As such, computing device 110 can learn audio that corresponds to afault (e.g., a particular type of fault) in a user (e.g., technician)supervised manner. That is, computing device 110 can capture thetechnician's expert knowledge in determining whether noise in theaudible range corresponds to a fault. For example, computing device 110can use supervised machine learning to classify audio as faults, andbuild an audio corpus (e.g., database of audio files) for differentclasses of faults.

Computing device 110 can then use this knowledge to detect (e.g., in anunsupervised manner) subsequent faults that may occur in HVAC equipment102-1, 102-2, . . . , 102-N. Further, computing device 110 can use thisknowledge to detect faults that may occur in similar equipment indifferent facilities, as will be further described herein. That is,computing device 110 can distribute the technician's expert knowledgeacross similar equipment in different facilities. As such, computingdevice 110 can learn fault patterns in the equipment over time via crowdsourcing for identification of anomalous audio, such that computingdevice 110 can accurately detect subsequent (e.g., future) faults in theequipment in a quicker and more efficient manner, without having to bepre-configured.

For example, audio sensors 104-1, 104-2, . . . , 104-M can sense theaudio that occurs during an additional (e.g., subsequent) eventassociated with the operation of HVAC equipment 102-1, 102-2, . . . ,102-N, and send this additional audio to computing device 106 (e.g.,computing device 106 can receive this additional audio from the audiosensors). Computing device 106 can then send, via network 108, thisadditional audio to computing device 110.

Computing device 110 can then determine whether the additional eventcorresponds to a fault occurring in the equipment based, at least inpart, on the input previously received from the user (e.g., the inputindicating the audio sensed during the previous event corresponded to afault). For example, computing device 110 can compare the audio sensedduring the additional event (e.g., the profile of the audio sensedduring the additional event) to the audio (e.g., to the profile of theaudio, stored by computing device 110) previously indicated by the useras corresponding to a fault to determine whether the audio (e.g., theprofile of the audio) sensed during the additional event matches theaudio (e.g., the profile of the audio) previously indicated ascorresponding to the fault. If the comparison indicates a match,computing device 110 can determine that that the additional eventcorresponds to a fault (e.g., the same type of fault that the previousevent corresponded to).

Upon determining the additional event corresponds to the fault (e.g.,upon the comparison indicating a match), computing device 110 canprovide an alert to the user of computing device 110 indicating thefault is occurring and that the equipment may need servicing. If,however, the comparison does not indicate a match (e.g., therebyindicating that the additional event does not correspond to the sametype of fault that the previous event corresponded to), computing device110 can provide the audio sensed during the additional event to the userto determine whether the additional event corresponds to a fault (e.g.,a different type of fault) occurring in the equipment, in a manneranalogous to that previously described herein for the audio sensedduring the previous event.

Upon the user determining the additional event corresponds to a faultoccurring in the equipment, computing device 110 can receive from theuser an input indicating that the additional event corresponds to thefault, in a manner analogous to that previously described herein for theprevious event. As such, computing device 110 can learn that theadditional event corresponds to the fault, and store the audio sensedduring the additional event accordingly. Computing device 110 can thenuse this additional knowledge in an analogous manner to detectsubsequent faults that may occur in HVAC equipment 102-1, 102-2, . . . ,102-N. This process can continue to be performed throughout theoperation of HVAC equipment 102-1, 102-2, . . . , 102-N, such thatcomputing device 110 can continue to learn and detect additional faults(e.g., additional types of faults) during the operation of the equipmentin a proactive manner (e.g., such that computing device 110 can continueto learn fault patterns in the equipment over time via crowd sourcingfor anomalous audio, such that computing device 110 can continue toaccurately detect future faults in the equipment in a quicker and moreefficient manner, without having to be pre-configured).

Further, this process can be used to learn and detect faults (e.g.,fault patterns) that occur in the equipment of different HVAC equipmentacross multiple facilities via crowd sourcing for identification ofanomalous audio, such that computing device 110 can accurately detectfaults across multiple facilities without having to be pre-configured.For example, although not shown in FIG. 1 for simplicity and not toobscure embodiments of the present disclosure, system 100 may includeadditional audio sensors located within an additional facility that cansense audio that occurs during an event associated with the operation ofthe equipment of the HVAC system of that facility. System 100 may alsoinclude an additional computing device located at the additionalfacility that can receive the audio sensed by the additional audiosensors, and send, via network 108, this audio to computing device 110.

Computing device 110 can then determine whether the event associatedwith the HVAC equipment of the additional facility corresponds to afault occurring in the equipment based, at least in part, on the inputpreviously received from the user (e.g., the input indicating the audiosensed during the event(s) associated with the operation of HVACequipment 102-1, 102-2, . . . , 102-N corresponded to a fault(s)). Forexample, computing device 110 can compare the audio (e.g., the profileof the audio) sensed during the event associated with the HVAC equipmentof the additional facility to the audio (e.g., to the profile of theaudio, stored by computing device 110) sensed during the event(s)associated with the operation of HVAC equipment 102-1, 102-2, . . . ,102-N previously indicated by the user as corresponding to a fault(s) todetermine whether the audio sensed during the event associated with theHVAC equipment of the additional facility matches the audio sensedduring the event(s) associated with the operation of HVAC equipment102-1, 102-2, . . . , 102-N previously indicated as corresponding to thefault(s). If the comparison indicates a match, computing device 110 candetermine that that the event associated with the HVAC equipment of theadditional facility corresponds to the matching fault.

Upon determining the event associated with the HVAC equipment of theadditional facility corresponds to a fault (e.g., upon the comparisonindicating a match), computing device 110 can provide an alert to theuser of computing device 110 indicating the fault is occurring and thatthe equipment may need servicing. If, however, the comparison does notindicate a match (e.g., thereby indicating that the event associatedwith the HVAC equipment of the additional facility does not correspondto the same type(s) of fault(s) that the event(s) associated with theoperation of HVAC equipment 102-1, 102-2, . . . , 102-N correspondedto), computing device 110 can provide the audio sensed during the eventassociated with the HVAC equipment of the additional facility to theuser to determine whether that event corresponds to a fault (e.g., adifferent type of fault) occurring in the equipment, in a manneranalogous to that previously described herein. Upon the user determiningthat event corresponds to a fault occurring in the equipment of theaddition facility, computing device 110 can receive from the user aninput indicating that event corresponds to the fault, in a manneranalogous to that previously described herein for the previous event. Assuch, computing device 110 can use this knowledge to detect subsequentfaults that may occur in HVAC equipment across multiple facilities(e.g., environments).

As an additional example, computing device 106 can receive the audiosensed by audio sensors 104-1, 104-2, . . . , 104-M during an eventassociated with the HVAC system of the facility. The event may be, forinstance, a safety, security, or maintenance event associated with theHVAC system, including the command and control of the HVAC system (e.g.,of HVAC equipment 102-1, 102-2, . . . , 102-N) during normal operation,observation of a mandatory safety procedure during operation of the HVACsystem, a performance degradation check of the HVAC system, performanceof repair work on the HVAC system, performance of upgrades on the HVACsystem, calibration of the HVAC system, a security breach, or anoccurrence of a safety violation, among others. The audio may include,for instance, voice commands, questions, and/or observations spoken by,and/or interactions between, individuals associated with (e.g., involvedin) the event.

Computing device 106 can identify the event based, at least in part, onthe audio sensed by the audio sensors during the event, and send, vianetwork 108, the identification of the event to computing device 110(e.g., computing device 110 can receive the identification of the eventfrom computing device 106 via network 108). For instance, computingdevice 106 can categorize and localize the source of the audio todetermine which type of event the audio is relevant to. Further,computing device 106 can distinguish between audio generated by the HVACequipment and human-generated audio to identify the event.

Computing device 110 can then initiate a response to the event based, atleast in part, on the identification of the event. For instance,computing device 110 can determine actions for the HVAC system (e.g.,the control system of the HVAC system) to take in response to theidentified event, and automatically instruct (e.g., trigger) the HVACsystem to take the determined actions, or provide (e.g., recommend) thedetermined actions to the user (e.g., technician) of computing device110.

As such, computing devices 106 and 110 can learn audio that correspondsto an event (e.g., a particular type of event), and the appropriateresponse to that event (e.g., to that type of event). Computing devices106 and 110 can then use this knowledge (e.g., self-calibrate) toidentify and respond to (e.g., in an unsupervised manner) subsequentevents associated with the HVAC system.

For example, audio sensors 104-1, 104-2, . . . , 104-M can sense theaudio that occurs during an additional (e.g., subsequent) eventassociated with the HVAC system, and send this additional audio tocomputing device 106 (e.g., computing device 106 can receive thisadditional audio from the audio sensors). Computing device 106 canidentify the additional event based, at least in part, on the audiosensed by the audio sensors during the additional event and theidentification of the previous event.

For example, computing device 106 can compare the audio (e.g., theprofile of the audio) sensed during the additional event associated withthe HVAC system to the audio (e.g., to the profile of the audio) sensedduring the previous event associated with the HVAC system previouslyidentified by computing device 106 to determine whether the audio sensedduring the additional event matches the audio sensed during the previousevent identified by computing device 106. If the comparison indicates amatch, computing device 106 can determine that that the additional eventassociated with the HVAC system corresponds to the same type of event asthe previously identified event, and identify the additional eventaccordingly. If the comparison does not indicate a match, the additionalevent may correspond to a different type of event, and computing device106 can identify the additional event accordingly.

Computing device 106 can then send, via network 108, the identificationof the additional event to computing device 110 (e.g., computing device110 can receive the identification of the additional event fromcomputing device 106 via network 108). Computing device 110 can theninitiate a response to the event based, at least in part, on theidentification of the additional event and the response initiated to thepreviously identified event. For example, if the identification of theadditional event indicates that the additional event is the same type ofevent as the previously identified event, computing device 110 caninitiate the same response that was initiated for the previouslyidentified event; if the identification of the additional eventindicates that the additional event is a different type of event thanthe previously identified event, computing device 110 can initiate aresponse for that different type of event. This process can continue tobe performed throughout the operation of the HVAC system, such thatcomputing devices 106 and 110 can continue to identify and respond tosubsequent events associated with the HVAC system in a proactive manner.

FIG. 2 illustrates a conceptual example of a method 201 for monitoringindustrial (e.g., HVAC) equipment using audio in accordance with anembodiment of the present disclosure. Method 201 can be performed by,for example, system 100 previously described in connection with FIG. 1.For example, the portion of method 201 shown on the facility side ofFIG. 2 can be performed by computing device 106 previously described inconnection with FIG. 1, and the portion of method 201 shown on theremote side of FIG. 2 can be performed by computing device 110previously described in connection with FIG. 1.

As shown in FIG. 2, sensed (e.g., raw) audio can be received at block220. The sensed audio can be received, for example, from audio sensors104-1, 104-2, . . . , 104-M previously described in connection with FIG.1 (e.g., the sensed audio can be analogous to the sensed audiopreviously described in connection with FIG. 1).

As shown in FIG. 2, a plurality of features can be extracted from thesensed audio at block 222, and the data associated with the extractedfeatures can be clustered at block 224. The extracted features, and themanner in which the features are extracted, can be analogous to thatpreviously described in connection with FIG. 1.

At block 226, an analytic module (e.g, an unsupervised analytic module)can determine whether any portion of the sensed audio is anomalous. Thisdetermination can be made in a manner analogous to that previouslydescribed in connection with FIG. 1. For instance, anomalous audio canbe detected using unsupervised (e.g., automatic) machine learning aspart of edge analytics, as previously described in connection withFIG. 1. As used herein, a “module” can include computer readableinstructions that can be executed by a processing resource (e.g.,processor) to perform a particular function. A module can also includehardware, firmware, and/or logic that can perform a particular function.

Upon a portion of the sensed audio being determined to be anomalous,that portion of audio can be sent from the facility side of method 201(e.g., from computing device 106) to the remote side of method 201(e.g., to computing device 110), in a manner analogous to thatpreviously described in connection with FIG. 1 (e.g., via network 108).It can then be determined, on the remote side of method 201, whether theanomalous portion of the audio corresponds to a fault using eithertraining mode or test mode, as previously described herein in connectionwith FIG. 1.

For example, in training mode, the anomalous portion of the audio can beplayed to a user at block 228 to determine whether the anomalous audioportion corresponds to a fault, as previously described in connectionwith FIG. 1. Upon the user determining the anomalous audio portioncorresponds to a fault, the user may label the anomalous audio portionas corresponding to the fault. For example, the user may classify thefault as belonging to one of fault classes 230-1, 230-2, . . . , 230-P,as shown in FIG. 1. The labelled (e.g. classified) anomalous audioportion may then be stored at block 232 by an analytic module (e.g., asupervised analytic module) for use in detecting subsequent faults thatmay occur during test mode. In such a manner, supervised machinelearning can be used to classify anomalous audio as faults, and build anaudio corpus (e.g., database of audio files) for different classes offaults.

During training mode, the anomalous portion of the audio can be inputinto the analytic module at block 232 to determine whether the anomalousaudio portion corresponds to a fault. For example, the analytic modulecan compare the anomalous portion of the audio to previous audioportions determined to be (e.g., labelled as) anomalous by the user atblock 228, in a manner analogous to that previously described inconnection with FIG. 1.

FIG. 3 illustrates an example display (e.g., a screen shot) 340 ofalerts 342-1 through 342-10 (collectively referred to herein as alerts342) and alert 344 of anomalous audio portions provided (e.g.,displayed) to a user (e.g., technician) in accordance with an embodimentof the present disclosure. The display can be provided to the user by,for example, computing device 110 previously described in connectionwith FIG. 1.

As shown in FIG. 3, display 340 includes a graphical representation(e.g., timeline) of the magnitude of the audio sensed during theoperation of equipment (e.g., an AHU) of an HVAC system of a facilitythroughout a particular day. As shown in FIG. 3, the graphicalrepresentation includes alerts 342 and 344 that each correspond to theoccurrence of a different anomalous portion of the audio (e.g., to thetime when each respective anomalous portion of the audio was sensed).For instance, alert 342-1 corresponds to an anomalous portion of theaudio sensed around 6:45, alert 342-2 corresponds to an anomalousportion of the audio sensed around 8:30, etc.

In the example illustrated in FIG. 3, alerts 342 are known alerts. Forinstance, the anomalous portions of audio that correspond to alerts 342correspond to (e.g., match) the audio of previously identified faults,as previously described herein. Further, in the example illustrated inFIG. 3, alert 344 is an unknown alert. For instance, the anomalousportion of audio that corresponds to alert 344 does not correspond to(e.g., does not match) the audio of any previously identified faults, aspreviously described herein.

Further, although not shown in FIG. 3 for clarity and so as not toobscure embodiments of the present disclosure, display 340 can include amechanism (e.g., a playback feature) for the user to listen to theanomalous portion of the audio. Display 340 may also include a mechanism(e.g., selectable buttons) for the user to identify (e.g., select)whether the anomalous portion of the audio corresponds to a fault or afalse alarm, and a mechanism (e.g., text input field) for the user toadd comments and/or remarks about the fault.

FIG. 4 illustrates an example of a computing device 450 for monitoringindustrial (e.g., HVAC) equipment using audio in accordance with anembodiment of the present disclosure. Computing device 450 can be anexample of, for instance, computing devices 106 and 110 previouslydescribed in connection with FIG. 1.

Computing device 450 can be, for example, a laptop computer, a desktopcomputer, or a mobile device (e.g., smart phone, tablet, PDA, etc.).However, embodiments of the present disclosure are not limited to aparticular type of computing device.

As shown in FIG. 4, computing device 450 can include a processor 452 anda memory 454. Memory 454 can be any type of storage medium that can beaccessed by processor 452 to perform various examples of the presentdisclosure. For example, memory 454 can be a non-transitory computerreadable medium having computer readable instructions (e.g., computerprogram instructions) stored thereon that are executable by processor454 to monitor equipment of HVAC system using audio in accordance withthe present disclosure. That is, processor 452 can execute theexecutable instructions stored in memory 454 to monitor equipment of anHVAC system using audio in accordance with the present disclosure.

Memory 454 can be volatile or nonvolatile memory. Memory 454 can also beremovable (e.g., portable) memory, or non-removable (e.g., internal)memory. For example, memory 454 can be random access memory (RAM) (e.g.,dynamic random access memory (DRAM), resistive random access memory(RRAM), and/or phase change random access memory (PCRAM)), read-onlymemory (ROM) (e.g., electrically erasable programmable read-only memory(EEPROM) and/or compact-disk read-only memory (CD-ROM)), flash memory, alaser disk, a digital versatile disk (DVD) or other optical diskstorage, and/or a magnetic medium such as magnetic cassettes, tapes, ordisks, among other types of memory.

Further, although memory 454 is illustrated as being located incomputing device 450, embodiments of the present disclosure are not solimited. For example, memory 454 can also be located internal to anothercomputing resource (e.g., enabling computer readable instructions to bedownloaded over the Internet or another wired or wireless connection).

As shown in FIG. 4, computing device 450 can include a user interface456. A user (e.g., operator) of computing device 450, such as, forinstance, a technician of an HVAC system, can interact with computingdevice 450 via user interface 456. For example, user interface 456 canprovide (e.g., display) information to and/or receive information from(e.g., input by) the user of computing device 450.

In some embodiments, user interface 456 can be a graphical userinterface (GUI) that can include a display (e.g., a screen) that canprovide and/or receive information to and/or from the user of computingdevice 450. The display can be, for instance, a touch-screen (e.g., theGUI can include touch-screen capabilities). As an additional example,user interface 456 can include a keyboard and/or mouse the user can useto input information into computing device 460, and/or a speaker thatcan play audio to the user. Embodiments of the present disclosure,however, are not limited to a particular type(s) of user interface.

Although specific embodiments have been illustrated and describedherein, those of ordinary skill in the art will appreciate that anyarrangement calculated to achieve the same techniques can be substitutedfor the specific embodiments shown. This disclosure is intended to coverany and all adaptations or variations of various embodiments of thedisclosure.

It is to be understood that the above description has been made in anillustrative fashion, and not a restrictive one. Combination of theabove embodiments, and other embodiments not specifically describedherein will be apparent to those of skill in the art upon reviewing theabove description.

The scope of the various embodiments of the disclosure includes anyother applications in which the above structures and methods are used.Therefore, the scope of various embodiments of the disclosure should bedetermined with reference to the appended claims, along with the fullrange of equivalents to which such claims are entitled.

In the foregoing Detailed Description, various features are groupedtogether in example embodiments illustrated in the figures for thepurpose of streamlining the disclosure. This method of disclosure is notto be interpreted as reflecting an intention that the embodiments of thedisclosure require more features than are expressly recited in eachclaim.

Rather, as the following claims reflect, inventive subject matter liesin less than all features of a single disclosed embodiment. Thus, thefollowing claims are hereby incorporated into the Detailed Description,with each claim standing on its own as a separate embodiment.

What is claimed is:
 1. A method for monitoring industrial equipment in afacility, comprising: receiving, from an audio sensor, an audio streamsensed by the audio sensor of an operation of industrial equipment inthe facility; sending at least part of the audio stream to a machinelearning module, wherein the machine learning module, trained by one ormore knowledge experts, determines whether the audio stream isindicative of a fault in the industrial equipment; and providing analert to a user when the machine learning module determines that theaudio stream is indicative of a fault in the industrial equipment. 2.The method of claim 1, wherein the machine learning module has atraining mode and a test mode, wherein the machine learning module istrained by the knowledge expert in the training mode.
 3. The method ofclaim 1, wherein the machine learning module is trained to classify theaudio stream into at least one of a plurality of differentclassifications.
 4. The method of claim 3, wherein at least one of theplurality of different classifications corresponds to a faultclassification.
 5. The method of claim 3, wherein at least one of theplurality of different classifications corresponds to a non-faultclassification.
 6. The method of claim 3, wherein the machine learningmodule comprises an audio corpus of different classifications.
 7. Themethod of claim 6, wherein the machine learning module is configured tocompare at least part of the audio stream to the audio corpus ofdifferent classifications to identify one or more matchingclassifications, if any.
 8. The method of claim 7, wherein when one ormore matching classifications are not identified, sending the audiostream to a knowledge expert for classification, and then incorporatingthe audio stream and the resulting classification to the audio corpus.9. The method of claim 7, wherein the alert identifies the one or morematching fault classifications when one or more matching faultclassifications are identified.
 10. The method of claim 1, wherein theaudio corpus of different classifications comprises a database of audiofiles obtained from industrial equipment located in a plurality ofdifferent facilities.
 11. A method for monitoring industrial equipmentin a facility, comprising: receiving, from an audio sensor, an audiostream sensed by the audio sensor of an operation of industrialequipment in the facility; sending at least part of the audio stream toa machine learning module; the machine learning module comparing thereceived audio stream to an audio corpus to attempt to classify theaudio stream into at least one of a plurality of differentclassifications; when the audio stream is classified by the machinelearning module into a fault classification, provide an alert to a user;and when the audio stream cannot be classified by the machine learningmodule: sending the audio stream to a knowledge expert forclassification; receiving a classification of the audio stream from theknowledge expert; and adding at least one or more characteristics of theaudio stream and the resulting classification to the audio corpus. 12.The method of claim 11, wherein at least one of the plurality ofdifferent classifications corresponds to the fault classification. 13.The method of claim 12, wherein at least one of the plurality ofdifferent classifications corresponds to a non-fault classification. 14.The method of claim 11, wherein the alert identifies the faultclassification in the alert.
 15. The method of claim 11, wherein theaudio corpus comprises a database of audio files obtained fromindustrial equipment located in a plurality of different facilities. 16.A method for monitoring industrial equipment in a facility, the methodcomprising: receiving, from an audio sensor, an audio stream sensed bythe audio sensor of an operation of industrial equipment in thefacility; sending at least part of the audio stream to a machinelearning module, wherein the machine learning module accesses an audiocorpus that includes a database of audio files obtained from industrialequipment located in a plurality of different facilities, wherein eachof the audio files is classified into one or more of a plurality ofclassifications; the machine learning module comparing at least part ofthe audio stream to the audio corpus to identify one or more matchingclassifications, if any; and when a matching classification isidentified by the machine learning module, and when the matchingclassification is a fault classification, providing an alert to notify auser of a possible fault in the industrial equipment.
 17. The method ofclaim 16, wherein: when a matching classification is not identified bythe machine learning module, sending the audio stream to a knowledgeexpert for classification, and then adding at least one or morecharacteristics of the audio stream and the resulting classification tothe audio corpus.
 18. The method of claim 16, wherein the machinelearning module is hosted on a cloud based service that services each ofthe plurality of different facilities.
 19. The method of claim 18,further comprising: processing the audio stream sensed by the audiosensor to determine an anomalous portion of the audio stream; andwherein the at least part of the audio stream that is sent to themachine learning module includes at least part of the audio stream thatis determined to be anomalous, and wherein at least part of the audiostream that is not determined to be anomalous is not sent to the machinelearning module.
 20. The method of claim 19, wherein the audio stream isprocessed to determine the anomalous portion of the audio stream via acontroller located in the facility.